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Whot is The Nation's Report Card? 

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is a nationally 
representative and continuing assessment of what America’s students know and can do in various subject areas. 

Since 1969, assessments have been conducted periodically in reading, mathematics, science, writing, history, 
geography, and other fields. By making objective information on student performance available to policymakers at 
the national, state, and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress 
of education. Only information related to academic achievement is collected under this program. NAEP guarantees 
the privacy of individual students and their families. 

NAEP is a congressionally mandated project of the National Center for Education Statistics within the Institute 
of Education Sciences of the U.S. Department of Education. The Commissioner of Education Statistics is 
responsible, by law, for carrying out the NAEP project through competitive awards to qualified organizations. 

In 1988, Congress established the National Assessment Governing Board (NAGB) to oversee and set policy for 
NAEP. The Board is responsible for: selecting the subject areas to be assessed; setting appropriate student 
achievement levels; developing assessment objectives and test specifications; developing a process for the review of 
the assessment; designing the assessment methodology; developing guidelines for reporting and disseminating 
NAEP results; developing standards and procedures for interstate, regional, and national comparisons; determining 
the appropriateness of all assessment items and ensuring the assessment items are free from bias and are secular, 
neutral, and non-ideological; taking actions to improve the form, content, use, and reporting of results of the National 
Assessment; and planning and executing the initial public release of National Assessment of Educational Progress 
reports. 
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xecutive Summary 

The National Assessment of Educational Progress (NAEP) 
is an ongoing nationally representative sample survey of 
student achievement in core subject areas. Authorized by 
Congress and administered by the National Center for 
Education Statistics (NCES) within the Institute of 
Education Sciences of the U.S. Department of Education, 
NAEP regularly reports to the public on the educational 
progress of fourth-, eighth-, and twelfth-grade students. 

This report presents the results of the NAEP 2002 reading 
assessment for the nation at grades 4, 8, and 12 and for 
participating states and other jurisdictions at grades 4 and 8. 
Assessment results are described in terms of students’ 
average reading score on a 0—500 scale and in terms of the 
percentage of students attaining each of three achievement 
levels: Bask ; Proficient, and Advanced. 

The achievement levels are performance standards adopted 
by the National Assessment Governing Board (NAGB) as 
part of its statutory responsibilities. The achievement levels 
are a collective judgment of what students should know and 
be able to do for each grade tested. As provided by law, 
NCES, upon review of a congressionally mandated 
evaluation of NAEP, determined that the achievement levels 
are to be used on a trial basis and should be interpreted with 
caution. However, both NCES and the Board believe these 
performance standards are useful for understanding trends in 
student achievement. They have been widely used by national 
and state officials and others as a common yardstick of 
academic performance. 
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The results presented in this report are 
based on representative samples of students 
for the nation and for participating states 
and other jurisdictions. Approximately 
270,000 students from 11,000 schools were 
assessed. The national results reflect the 
performance of students attending both 
public and nonpublic schools, while the state 
and jurisdiction results reflect only die perfor- 
mance of students attending public schools. 

In addition to providing average scores 
and achievement level performance in 
reading for the nation and states and other 
jurisdictions, this report provides results for 
subgroups of students defined by various 
background characteristics. A summary of 
major findings from the NAEP 2002 
assessment is presented on the follow- 
ing pages. Comparisons are made to 
results from previous years in which 
the assessment was administered. In 
addition to the 2002 results, national 
results are reported from the 1992, 

1994, 1998, and 2000 (fourth-grade 
only) assessments. State and/or jurisdic- 
tion results are also reported from the 
1992, 1994, and 1998 assessments at 
grade 4 and from the 1998 assessment 
at grade 8. The more recent results 
(those from 1998 or later) are based on 
administration procedures in which 
testing accommodations were permit- 
ted for students with disabilities and 
limited English proficient students. 
Accommodations were not permitted in 
earlier assessments. Comparisons 
between results from 2002 and those from 
assessment years in which both types of 
administration procedures were used (1998 
at all three grades and 2000 at grade 4 only) 
are discussed in this executive summary 
based on the results when accommodations 
were permitted. Changes in student perfor- 



mance across years or differences between 
groups of students in 2002 are discussed 
only if they have been determined to be 
statistically significant. 

Overall Reading Results for 
the Nation and the States 

Reading Results for the Nation 
At grade 4 

□ The fourth-grade average score in 
2002 was higher than in 1994, 1998 
and 2000, but was not found to be 
significantly different from 1992. 

m Scores at the 10th, 25th, and 50th 
percentiles were higher in 2002 than 
in 1998 and 2000 but were not found 
to be significantly different from 
1992. The score at the 75th percentile 
was higher than in 1992, indicating 
improvement for higher performing 
fourth-grade students. 

SS The percentage of fourth-graders 
who performed at or above the Basic 
level in 2002 was higher than in 
1994, 1998, and 2000 but was not 
found to be significantly different 
from 1992. The percentage at or 
above Proficient was higher in 2002 
than in 1992 and 1998. 

At grade 8 

© The eighth-grade average score in 2002 
was higher than in 1992 and 1994. 

□ Scores were higher in 2002 than in 
1992 for all but the highest 
performing eighth -grade students 

(at the 10th, 25th, 50th, and 75th 
percentiles). 

H The percentage of eighth -graders who 
performed at or above Basic was higher in 
2002 than in all previous assessment 
years, and the percentage at or above 
Proficient was higher than in 1992 and 1994. 
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M grade 112 

□ The twelfth-grade average score in 2002 
was lower than in 1992 and 1998. 

□ At grade 12, declines in performance 
since 1992 were evident across most 
of the score distribution (10th, 25th, 
50th, and 75th percentiles). 

□ The percentages of twelfth-graders 
who performed at or above the Basic ' 
and Proficient levels decreased 
between 1998 and 2002, and thus fell 
below levels seen in 1992. 

^esodorag Results for foe Skates 
samidl ©foer Jurisdictions 

Results from the 2002 assessment are 
reported for 48 states and other juris- 
dictions at grade 4, and 47 states and 
other jurisdictions at grade 8. An addi- 
tional two states at grade 4 and three 
states at grade 8 participated in the 
2002 assessment, but did not meet 
minimum participation guidelines for 
reporting results. Results for public- 
school students only are reported at the 
state or jurisdiction level. (Throughout 
this summary, the term jurisdiction is 
used to refer to the states, territories, 
and Department of Defense schools 
that participated in the NAEP reading 
assessments). 




□ Among the 40 jurisdictions that 
participated in both the 1992 and 2002 
assessments, fourth-graders’ average 
scores increased in 15 jurisdictions and 
decreased in 2 jurisdictions. The 
percentage of students at or above 
Proficient increased in 17 of the 
jurisdictions during the same time period. 

□ Connecticut, Massachusetts, and Ver- 
mont were among the highest-perform- 
ing states at grade 4 in 2002. The average 
scores for fourth-graders in Connecticut 
and Vermont were not found to be 
significantly different from each other, 
and fourth-graders in both states were 
outperformed on average by only those 
in Massachusetts. 

M grade 8 

H Among the 37 jurisdictions that partici- 
pated in both the 1998 and 2002 assess- 
ments, eighth-graders’ average scores 
increased in 10 jurisdictions and de- 
creased in 5 jurisdictions. The percentage 
of students at or above Proficient increased 
in 5 jurisdictions and declined in 1 juris- 
diction during the same time period. 

H The Department of Defense domestic 
and overseas schools, Vermont, and 
Massachusetts were among the 
highest-performing jurisdictions at 
grade 8 in 2002. The average scores 
for eighth-graders in these jurisdic- 
tions were not found to differ signifi- 
cantly from each other. 
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National and State 
Reading Results for 
Student Subgroups 

In addition to overall results for the nation 
and for the states and jurisdictions, NAEP 
reports on the performance of various 
subgroups of students. In interpreting these 
data, readers are reminded that the relation- 
ship between contextual variables and 
student performance is not necessarily 
causal. There are many factors that may play 
a role in student achievement in a particular 
subject area. 

Nationcsll 

Gemder 

□ The average scores of male and of 
female fourth-graders were higher in 
2002 than in 1998 but were not 
found to be significantly different 
from the scores in 1992. Average 
scores of male and female eighth- 
graders were higher in 2002 than in 
1992 and 1994. In contrast, the 
average scores of male and female 
twelfth-graders were lower in 2002 
than in 1992 and 1998. 

□ In 2002, females had higher average 
reading scores than males at all three 
grades. 

□ The gap between average scores for 
male and female fourth-graders in 
2002 was not found to be signifi- 
cantly different from that in 1992. At 
grade 8, the gap was smaller in 2002 
than in all previous assessment years. 
The gap at grade 12, however, was 
wider in 2002 than it had been in 
1992. 

□ The percentages of female fourth-, 
eighth-, and twelfth-graders at or 
above Proficient in 2002 were not 
found to differ significantly from those in 



1992. The percentage of male eighth- 
graders at or above Proficient was higher in 
2002 than in 1992, and the percentage of 
twelfth-grade males was lower in 2002 
than in 1992. 




□ At grades 4 and 8, both White and Black 
students had higher average scores in 
2002 than in 1992. Similar increases 
across the decade were seen for eighth- 
grade Hispanic students and fourth-grade 
Asian/Pacific Islander students. The 
average scores for White and Black 
twelfth-graders, however, declined during 
die same time period. 

□ In 2002, White students and Asian/ 
Pacific Islander students had higher 
average scores than Black and His- 
panic students, and White students 
outperformed Asian/Pacific Islander 
students at all three grades. American 
Indian/Alaska Native students had 
higher average scores than Black and 
Hispanic students at grade 4. 

□ In 2002, the score gap between 
White and Black fourth-graders was 
smaller than in 1994 and the gap 
between White and Hispanic fourth- 
graders was smaller than in 2000, but 
neither gap was found to be signifi- 
cantly different from 1992. No changes 
were detected in the gaps between 
White and Black students and be- 
tween White and Hispanic students 
at grades 8 and 12 since 1992. 

0 Percentages of students at or above 
Proficient were higher in 2002 than 
in 1992 for White, Black, and Asian/ 
Pacific Islander fourth-graders and 
for White and Black eighth-graders. 
The percentage of White twelfth -graders 
at or above Proficient was lower in 2002 
than in 1992. 
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Eligibility for 

IF free/ Reduced-Price Lunch 

The program providing free/reduced- 
price lunch is administered by the U.S. 
Department of Agriculture (USDA) for 
children near or below the poverty 
line. Eligibility is determined by the 
USDAs Income Eligibility Guidelines 
(http:/ /www.fns.usda.gov/cnd/ 
IEGs&NAPs/IEGs.htm). Reading results 
by this variable are only available back 
to 1998. 

03 Average scores increased between 1998 
and 2002 for fourth- and eighth -graders 
eligible for free/reduced-price lunch. No 
change was detected between 1998 and 
2002 in the average score for twelfth- 
graders who were eligible, while the 
score for students who were not eligible 
decreased. 

□ In 2002, at all three grades students who 
were eligible for free/reduced-price lunch 
had lower average scores than students 
who were not eligible. 

Me i I Participation 

Title I is a federally funded program 
that provides educational services to 
children who live in areas with high 
concentrations of low-income families. 
Because of recent changes in how the 
program is administered, comparisons 
to previous assessment-year results are 
not available. 

03 As was observed in previous assessments, 1 
students at all three grades who attended 
schools that received Title I funding had 
lower average reading scores in 2002 than 
students who attended schools that 
reported not receiving funds. 



Parents' Level of Education 

Eighth- and twelfth-grade students who 
participated in the NAEP reading assess- 
ment were asked to indicate the highest 
level of education completed by each 
parent. Information about parental educa- 
tion was not collected at grade 4. 

D At grade 8, average scores increased 
between 1992 and 2002 for students 
whose parents did not graduate from 
high school, as well as for students 
whose parents' highest level of 
education was either high school or 
college graduation. At grade 12, 
average scores in 2002 were lower 
than in 1992 regardless of parental 
education level. 

□ As seen in previous assessments, 2 
a positive relationship between 
student-reported parental education 
and student reading performance 
was observed in 2002 at grades 8 
and 12: the higher the parental 
education level, the higher the 
student's average reading score. 

Type of School 

H The average score for fourth-grade 
public-school students was higher in 
2002 than in 1994, 1998, and 2000 
but was not found to differ signifi- 
cantly from 1992. Eighth-graders 
attending public schools or Catholic 
schools had higher average scores in 
2002 than in 1992. Twelfth-graders 
attending public schools had lower 
scores in 2002 than in 1992 and 1998. 



1 Donahue, P. L., Voelkl, K. E., Campbell, J. R., and Mazzeo, J. (1999). The 1998 NAEP Reading Report 
Card for the Nation and the States (NCES 1999-500). Washington, DC: U.S. Department of Education, 
Office of Education Research and Improvement, National Center for Education Statistics. 

2 Ibid. 
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□ In 2002, at all three grades students who 
attended nonpublic schools had higher 
average reading scores than their peers 
who attended public schools. 




D Fourth-graders attending schools in 
central city or urban fringe/large 
town locations had higher average 
scores in 2002 than in 2000. (Results by 
type of location are not available prior to 
2000 at grade 4, or prior to 2002 at 
grades 8 and 12.) 

□ In 2002, at all three grades students in 
schools located in urban fringe/large 
town areas outperformed students in 
schools located in central city and rural 
areas. 

Statfe amid Jurisdiction Results 
Gender 

Among those jurisdictions that participated 

in both the 1998 and 2002 assessments, 

□ both male and female fourth-graders 5 
average scores increased in 13 juris- 
dictions: Delaware, Florida, Georgia, 
Hawaii, Massachusetts, Minnesota, 
North Carolina, Oregon, Utah, 
Virginia, Washington, District of 
Columbia, and Department of 
Defense domestic schools; 

□ both male and female eighth-graders 5 
average scores increased in two 
jurisdictions: Delaware and Florida. 



Race/Etlmkify 

Among those jurisdictions that participated 

in both the 1998 and 2002 assessments, 

□ average scores increased for at least three 
different racial/ethnic subgroups of 
fourth-graders in five jurisdictions: 
Delaware, Massachusetts, New York, 
Oregon, and Virginia. 

□ both White and Black eighth-graders 5 
average scores increased in three 
jurisdictions: Delaware, Florida, and 
Missouri. 

illigihiSsSy for 

Free/R&duced-Prke Lunch 

Among those jurisdictions that partici- 
pated in both the 1998 and 2002 assess- 
ments, 

□ average scores increased for both 
fourth-graders who were eligible and 
those who were not eligible for free/ 
reduced-price lunch in 14 jurisdic- 
tions: Arkansas, Delaware, Florida, 
Hawaii, Louisiana, Maryland, Massa- 
chusetts, New York, North Carolina, 
Oregon, South Carolina, Utah, Vir- 
ginia, and Washington. 

□ average scores increased for both 
eighth-graders who were eligible and 
those who were not eligible for free/ 
reduced-price lunch in five jurisdic- 
tions: Arkansas, Delaware, Missouri, 
Washington, and Department of 
Defense overseas schools. 
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Introduction 



Reading is the foundation for many learning endeavors and 
one important key to unlocking a world of possibilities and 
opportunities. It has always been viewed as one of the 
most important abilities that students learn and 
continuously develop throughout their years in elementary 
and secondary school. With passage of the No Child Ceft 
Behind Act of 2001, however, the nation placed new and 
even greater emphasis on ensuring that every student 
acquires the ability to read. 

This report presents major results from the 2002 National 
Assessment of Educational Progress (NAEP) reading 
assessment of the nation’s fourth-, eighth-, and twelfth- 
grade students. In addition, the report provides results for 
fourth- and eighth-grade students in states and other 
jurisdictions that participated in the 2002 assessment. 

The report is intended to inform educators, policymakers, 
parents, and the general public about students’ achievement 
in reading. In doing so, the report serves an important role 
in monitoring progress toward the nation’s goal of ensuring 
that no child is left behind. 

Overview of the 2002 National Assessment 
of Educational Progress in Reading 

For more than thirty years, NAEP has regularly collected, 
analyzed, and reported valid and reliable information about 
what American students know and can do in a variety of 
subject areas. As authorized by the U.S. Congress, NAEP 
assesses representative national samples of fourth-, eighth-, 
and twelfth-grade students. Since 1990, NAEP has also 
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assessed representative samples of fourth- 
and eighth-grade students in states and 
other jurisdictions that participate in the 
NAEP state-by-state assessments. NAEP is 
administered and overseen by the National 
Center for Education Statistics (NCES), 
which is one of three centers within the 
U.S. Department of Education’s Institute 
of Education Sciences. 

The content of all NAEP assessments is 
determined by subject-area frameworks 
that are developed by the National Assess- 
ment Governing Board (NAGB) in a 
comprehensive process involving a broad 
spectrum of interested parties, including 
teachers, curriculum specialists, subject- 
matter specialists, school administrators, 
parents, and members of the general 
public. The framework for the 2002 NAEP 
reading assessment has guided develop- 
ment of the NAEP reading assessments 
since 1992. 

The 2002 assessment was conducted at 
grades 4, 8, and 12 nationally, and at grades 
4 and 8 within the states and other jurisdic- 
tions that participated in the state-level 
assessment. Throughout this report, results 
from the 2002 assessment are compared to 
those from previous years. Trends in 
students’ reading achievement can be 
examined by comparing results from the 
most current assessment with results of 
earlier assessment administrations for 
same-grade students; such comparisons of 
national results are made at all three grade 
levels. Also included are comparisons of 
results for states and jurisdictions that 
participated in both 2002 and previous ♦ 
state-level assessment administrations. 



The reading assessment administered in 
2002 was the same as that given in 1992 to 
fourth-, eighth-, and twelfth-graders 
nationally — and again in 1994 and 1998. 

In addition, a national assessment of 
fourth-graders only was conducted in 2000. 
State-level assessments using the same test 
as that used nationally were conducted at 
grade 4 in 1992, 1994, and 1998. Similarly, 
a state-level assessment was conducted at 
grade 8 in 1998. 

Prior to 1998, administration procedures 
for NAEP reading assessments did not 
permit the use of accommodations (e.g., 
extra time, individual rather than group 
administration) for special needs students 
who could not participate without them. 

For the 1998 assessment, however, admin- 
istration procedures were introduced that 
allowed the use of accommodations by 
students with disabilities and limited 
English proficient students (see appendix 
A). A split-sample design was used in 1998 
at all three grades (and again in 2000 at 
grade 4) so that both administration proce- 
dures could be used during the same 
assessment, but with different samples of 
students. This made it possible to report 
trends in students’ reading achievement 
across all the assessment years and, at the 
same time, examine the effects on overall 
assessment results of including students 
assessed with accommodations. Based on 
an examination of how permitting accom- 
modations affected overall population 
results, it was decided that beginning with 
the 2002 assessment NAEP would use only 
one set of procedures — permitting the use 
of accommodations. 
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This change in administration procedures 
makes it possible for more students to be 
included in the assessments; however it 
also represents an important altering of 
procedures from previous assessments. The 
reader is encouraged to consider the differ- 
ence in accommodation procedures when 
interpreting comparisons between the two 
sets of results. During the period in which 
accommodations were not permitted, 
special needs students could only be 
included in the assessment if it was deter- 
mined by school staff that they could be 
assessed meaningfully without accommoda- 
tions. As a consequence, some students 
who would have been assessed in more 
recent years when accommodations were 
permitted may have been excluded from 
those earlier assessments. The charts and 
tables throughout this report distinguish 
between results from assessment years in 
which accommodations were not permitted 
and results from assessment years in which 
accommodations were permitted. 

In the tables and charts that display 
results across assessment years, all previous 
assessment results that were found to be 
significantly different from the 2002 results 
are marked with an asterisk (*). Two sets 
of results are presented for assessment 
years in which both administration proce- 
dures were used (accommodations not 
permitted and accommodations permitted). 
Both sets of results may also be notated, if 
found to be significantly different from 
2002. The text that accompanies these 
tables and charts indicates which previous 
assessment results were significantly 
different from 2002. Comparisons between 
the 2002 results, when accommodations 



were permitted, and the 1992 and 1994 
results, when they were not permitted, 
are discussed in the text. However, for 
previous assessment years with both 
accommodations-not-permitted results 
and accommodations-permitted results, 
the text describes comparisons only 
between the accommodations-permitted 
results and 2002. (See appendix A for 
further discussion of assessing students 
with disabilities and/or limited English 
proficient students.) 

Framework for the 1 992, 
1994, 1998, 2000, and 2002 
NAEP Reading Assessments 

The NAEP reading framework is the 
blueprint that has specified the content and 
guided the development of each NAEP 
reading assessment administered since 
1992. The framework resulted from a 
national process involving many organiza- 
tions concerned with reading education. 
This cooperative effort was managed by the 
Council of Chief State School Officers 
(CCSSO) and directed by NAGB. In 2002, 
the NAEP reading framework was updated 
to provide more explicit detail regarding the 
assessment design. 1 At that time, NAGB 
altered slightly some of the terms used to 
describe elements of the reading assess- 
ment. The following description of the 
NAEP reading framework incorporates 
these changes. It should be noted, however, 
that this updating of the framework does 
not represent a change in the content or 
design of the NAEP reading assessment. 

The framework is founded on research 
from the field of education that defines 
reading as an interactive and constructive 
process involving the reader, the text, and 



1 National Assessment Governing Board. (2002). Reading Framework for the 2003 National Assessment of Educational 
Progress . Washington, DC: Author. 
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the context of the reading experience. 
Reading involves the development of an 
understanding of text, thinking about text 
in different ways, and using a variety of 
text types for different purposes. For 
example, readers may read stories to enjoy 
and appreciate the human experience, study 
science texts to form new hypotheses about 
knowledge, or use directions to learn how 
to do something. 

Recognizing that readers vary their 
approach to reading according to the 
demands of any particular text, the frame- 



work specifies the assessment of reading in 
three contexts: reading for literary experi- 
ence, reading to gain information, and 
reading to perform a task. Each context for 
reading is associated with a range ot 
different types of texts that are included in 
the NAEP reading assessment. All three 
contexts for reading arc assessed at grades 
8 and 12, but reading to perform a task is 
not assessed at grade 4. The three contexts 
for reading as specified in the framework 
are described in figure 1.1. 



Figure 1 . 1 Descriptions of the three contexts for reading in the NAEP reading ossessment 



Contexts lor Rending 



Reading for Involves the reader in exploring themes , events , characters, settings, plots, actions, 

literary experience and the language of literary works . 

Various types of texts ore ossocioted with reading for literory experience, including novels, short 
stories, poems, ploys, legends, biogrophies, myths, ond folktales. 



R eading for Involves the engagement of the reader with aspects of the real world. 

information Reading for information is most commonly ossocioted with textbooks, primary ond secondary sources, 

newspopers ond mogozine articles, essoys, ond speeches. 



Reading to 
perform a task 



Involves reading in order to accomplish or do something. 

Practical text reod to perform o tosk moy include chorts, bus or troin schedules, directions for gomes 
or repoirs, dossroom or librory procedures, tox or insuronce forms, recipes, voter registration 
moteriols, mops, referendo, consumer warranties, or office memos. 



SOU RCE: National Assessment Governing Board. (2002). Reading Framewori r for the 2003 Hational Assessment of Educational Progress. Washington, DC : Author. 



As readers attempt to develop under- 
standing of text, they focus on general 
topics or themes, interpret and integrate 
ideas, make connections to background 
knowledge and experiences, and examine 
the content and structure of the text. The 
framework accounts for these different 
approaches to understanding text by 



specifying four “aspects of reading” that 
represent the types of comprehension 
questions asked of students. All four 
aspects of reading are assessed at all three 
grades within each context of reading 
described above. The four aspects of 
reading as specified in the framework are 
desenbed in figure 1.2. 
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Figure 1 .2 Descriptions of the four ospects of reoding in the NAEP reading assessment 



Aspects o! Reading 



Forming a To form a general understanding, the reader must consider the text as a whole 

general understanding 1 ond provide a global understanding of it. 

Students may be asked, for example, to demonstrate a general understanding by giving the topic 
of a passage, explaining the purpose of an article, or reflecting on the theme of a story. 



Developing 

interpretation 



To develop an interpretation, the reader must extend initial impressions to develop 
a more complete understanding of what was read. 

This process involves linking information across parts of a text as well as focusing on specific 
information. Questions that assess this aspect of reading include drawing inferences about 
the relationship of two pieces of information and providing evidence to determine the reason 
for an action. 



Making reader/text 
connections 2 



Examining content 
and structure 3 



To make reader/text connections, the reader must connect information in the text 
with knowledge and experience. 

This process might include applying ideas in the text to the real world. All student responses 
to these types of questions must be text-based to receive full-credit. 

Examining text content and structure requires critically evaluating, comparing and contrasting, 
and understanding the effect of such features as irony, humor, and organization. 

Questions used to assess this aspect of reading require readers to stand apart from the text, consider 
it objectively, and evaluate its quality and appropriateness. Questions ask readers to determine the 
usefulness of a text for a specific purpose, evaluate the language and textual elements, and think 
about the author's purpose and style. 



1 It* aspect of readrg was formerly referred to as 'formiig on inifid understanding' n previous versions of twKAffreaingfninewoA. 

? It* asped of readng was formerly refened to as 'personal refWtwn and response' n previous versions of tic HAEP readhg framework. 

3 Hits aspect of readmg was formerly refened to as 'demonstrating o critkd stance' in previous versions of the KAEP reodrg fnmework. 

SOURCE: Natkmd Assessment Governing Board. (2002). Reading Framework for the 2003 Hationoi Assessment of educational Progress. Washington, DC: Author. 



The 2002 NAEP Reading 
Assessment Instrument 

The NAEP reading assessment is the only 
federally authorized, ongoing, nationwide 
assessment of student reading achieve- 
ment. As such, it is necessary for the 
assessment to reflect the framework and 
expert perspectives on the measurement of 
reading comprehension. To that end, during 
the development process, the assessment 
undergoes stringent review by teachers and 
teacher educators, as well as by state 
officials and measurement specialists. All 
components of the assessment are evalu- 



ated for curricular relevance, developmen- 
tal appropriateness, and fairness concerns. 

The NAEP reading assessment measures 
understanding by prompting students to 
read passages and answer comprehension 
questions. The reading passages used in the 
NAEP assessment are drawn from the 
types of books and publications that 
students might encounter in school, in the 
library, or at home. NAEP assessment 
developers strive to replicate authentic 
reading experiences in the assessment items 
presented to student participants. The 
passages students are asked to read are 
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neither abridged nor contrived especially 
for the assessment. Instead, full-length 
reading selections are reprinted in test 
booklets to resemble as closely as possible 
die format of their original publication. To 
demonstrate their comprehension of these 
passages, students answer a combination of 
mulriple-choice and constructed-response 
questions. The multiple-choice questions 
include four options from which students 
are asked to select the best answer. The 
constructed-response questions require 
students to write their own responses. 

Short constructed-response questions can 
be completed in no more than a few sen- 
tences, while extended constructed-re- 
sponse questions may require students to 
provide responses as long as a paragraph or 
a full page. 

In order to ensure reliable and valid 
scoring of constructed-response questions, 
a unique scoring guide, describing the 
specific criteria for assigning a score level 
to each student’s response, is developed for 
each question. Expert scorers go through 
extensive training to understand how to 
apply these scoring criteria fairly and 
consistently. During the scoring process, 
scorers are consistently monitored to 
ensure that scoring standards are being 
applied appropriately and to ensure a high 
degree of scorer agreement (i.e., interrater 
reliability). In addition, for those con- 
structed-response questions that were used 
in previous assessments, monitoring of 
scorers includes checking to make sure that 
scoring standards remain consistent from 
year to year. 



At each grade, the entire reading assess- 
ment is divided into sections referred to as 
blocks. Each block contains at least one 
text and a related set of approximately 10 
to 12 comprehension questions (a combina- 
tion of multiple-choice and constructed- 
response). Most of the blocks are presented 
to students as 25-minute timed sections, 
but some are presented as 50-minute timed 
sections. The total number of blocks that 
comprise the NAEP reading assessment at 
each grade are as follows: 

Grade 4 — four 25-minute literary blocks 
and four 25-minute informative 
blocks; 

Grade 8 — three 25-minute literary blocks, 
three 25-minute informative 
blocks, three 25-minute task 
blocks, and one 50-minute 
informative block; 

Grade 12 — three 25-minute literary blocks, 
three 25-minute informative 
blocks, three 25-minute task 
blocks, and two 50-minute 
informative blocks. 

In order to minimize the burden on any 
individual student, NAEP uses a procedure 
referred to as matrix sampling in which an 
individual student is administered only a 
small portion of the entire assessment at 
any grade. For example, at grade 4, students 
are given a test booklet that contains only 
two 25-minute blocks. At grades 8 and 12, 
students are given a test booklet that 
contains either two 25-minute blocks or 
one 50-minute block. Because each block 
is administered to a representative sample 
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at each grade, the results can then be 
combined to produce average group and 
subgroup results based on the entire assess- 
ment. In addition to the two 25-minute 
blocks or one 50-minute block in each 
student’s test booklet, students are asked to 
complete two sections of background 
questions that ask about their background 
and home or school experiences related to 
reading achievement. In total, the time 
required for each student to participate in 
the NAEP reading assessment is no more 
than one hour. 

Description of School 
and Student Samples 

The NAEP 2002 reading assessment was 
administered to fourth-, eighth-, and 
twelfth -graders at the national level and to 
fourth- and eighth-graders at the state level. 
At the national level, results are reported 
for both public- and nonpublic-school 
students. At the state or jurisdiction level, 
results are reported only for public school 
students. In order to obtain a representative 
sample of students for reporting national 
and state or jurisdiction results, 
approximately 140,000 fourth-graders from 
5,500 schools, 115,000 eighth-graders from 
4,700 schools, and 15,000 twelfth-graders 
from 700 schools were sampled and 
assessed. In states that did not participate, 
a small sample of students proportionate to 
the state’s student enrollment was sampled 
and assessed. Each selected school that 
participated in the assessment and each 
student assessed represent a portion of the 
population of interest. For information on 
sample sizes and participation rates by state 
or jurisdiction, see tables A.4-A.6 in 
appendix A. 



Reporting the 
Assessment Results 

Results from the NAEP reading assessment 
are presented in two ways: as scale scores 
and as percentages of students attaining 
achievement levels. The scale scores, 
indicating how much students know and can 
do in reading, are presented as average scale 
scores and as scale scores at selected 
percentiles. The achievement level results 
indicate the degree to which student 
performance meets the standards set for 
what they should know and be able to do. 
Results are reported only for groups or 
subgroups of students; individual student 
performance cannot be reported based on 
the NAEP assessment. 

Average scale score results are based on 
the NAEP reading scale, which ranges from 
0 to 500. In order to calculate students’ 
average scores on the NAEP reading 
assessment, the analysis begins by deter- 
mining the percentages of students re- 
sponding correctly to each multiple-choice 
question and the percentages of students 
responding at each score level for the 
constructed-response questions. The 
analysis entails summarizing the results on 
separate subscales for each reading context 
(reading for literary experience, reading for 
information, and reading to perform a task) 
and then combining the separate scales to 
form a single composite reading scale. The 
relative contribution of each reading 
purpose at each grade is displayed in table 
1.1. (See appendix A for more information 
on scaling procedures.) 
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Table 1,1 Percentage weighting of the "context for reading" subscoles on the NAEP composite reading scale, 
grades 4, 8, and 12 





Reading for 
literary experience 


Reading for 
information 


Reading to 
perform a task 


Grade 4 


55 


45 


— 


Grade 8 


40 


40 


20 


Grade 12 


35 


45 


20 



— Not assessed at grade 4. 

SOURCE: Nationd Assessment Govemiig Board. (2002). Roofing framework for the 2003 Hatronoi Assessment of Educational Progress. Washington, DC: Author. 



Achievement level results are presented 
in terms of reading achievement levels as 
authorized by the NAEP legislation and 
adopted by NAGB. For each grade assessed, 
NAGB has adopted three achievement 
levels: Basic) Proficient , and Advanced. For 
reporting purposes, achievement level cut 
scores arc placed cm the reading scale, 
resulting in four ranges: below Basic ) Basic) 
Proficient , and Advanced. The achievement 
level results are then reported as percent- 
ages of students within each achievement 
level range, as well as the percentage of 
students at or above Basic and at or above 
Proficient. 



The Setting of 
Achievement Levels 

The 1988 NAEP legislation that created 
the National Assessment Governing Board 
directed the Board to identify “appropriate 
achievement goals . . . for each subject area” 
that NAEP measures. 2 The 2001 NAEP 
reauthorization reaffirmed many of the 
Board’s statutory responsibilities, including 
“developing appropriate student perfor- 
mance standards for each age and grade in 
each subject area to be tested under the 
National Assessment.” 3 In order to follow 
this directive and achieve the mandate of 
the 1988 statute “to improve the form and 



2 National Assessment of Educational Progress Improvement Act, Pub. L. No. 100-297, 20 U.S.C. § 1221 et seq. 
(1988). 

3 No Child Left Behind Act of 2001, Pub. L No. 107-110, 115 Stat. 1425 (2002). 




*) i 
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use of NAEP results,’' NAGB undertook 
the development of student performance 
standards (called “achievement levels”). 
Since 1990, the Board has adopted 
achievement levels in mathematics, read- 
ing, U.S. history, world geography, science, 
writing, and civics. 

The Board defined three levels for each 
grade: Basic , Proficient , and Advanced. The 
Basic level denotes partial mastery of the 
knowledge and skills that are fundamental 
for proficient work at a given grade. The 
Proficient level represents solid academic 
performance. Students reaching this level 
demonstrate competency over challenging 
subject matter. The Advanced level pre- 
sumes mastery of both the Basic and 



Proficient levels and represents superior 
performance, figure 1.3 presents the policy 
definitions of the achievement levels that 
apply across grades and subject areas. The 
policy definitions guided the development 
of the reading achievement levels, as well 
as the achievement levels established in all 
other subject areas assessed by NAEP. 
Adopting three levels of achievement for 
each grade signals the importance of: 
looking at more than one standard of 
performance. The Board believes, however, 
that all students should reach the Proficient 
level; the Basic level is not the desired goal, 
but rather represents partial mastery that is a 
step toward Proficient. 



Figure 1.3 Policy definitions of the three NAEP achievement levels 




Basic This level denotes partial mastery of prerequisite knowledge and skills that are fundamental for 

proficient work at each grade. 

Proficient This level represents solid academic performance for each grade assessed. Students reaching this 

level have demonstrated competency over challenging subject matter, including subject-matter 
knowledge, application of such knowledge to real-world situations, and analytical skills appropriate 
to the subject matter. 

Advanced This level signifies superior performance. 



SOURCt Notional Assessment Governing Boai (2002). Reading Framework (or the 2003 National Assessment o( Educational Progress, Washington, DC: Author. 
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The achievement levels in this report 
were adopted by the Board based on a 
standard-setting process designed and 
conducted under a contract with ACT, Inc. 
To develop these levels, ACT convened a 
cross section of educators and interested 
citizens from across the nation and asked 
them to judge what students should know 
and be able to do relative to a body of 
content reflected in the NAEP assessment 
framework for reading. This achievement 
level setting process was reviewed by an 
array of individuals including policymakers, 
representatives of professional organiza- 
tions, teachers, parents, and other members 
of the general public. Prior to adopting 
these levels of student achievement, 

NAGB engaged a large number of persons 
to comment on the recommended levels 
and to review the results. 

The results of the achievement level 
setting process, after NAGB’s approval, 
became a set of achievement level descrip- 
tions and a set of achievement level cut 
scores. The cut scores are the scores on 
the 0-500 NAEP reading scale that 
define the lower boundaries of Basic, 
Proficient, and Advanced performance 
levels at grades 4, 8, and 12. The Board 
established these reading achievement 
levels in 1992 based upon the reading 
assessment framework. These levels are 
used to describe student performance 
on the 1992, 1994, 1998, 2000, and 2002 
reading assessments. 



Reading Achievement Level 
Descriptions for Each Grade 

Specific definitions of the Basic, Proficient, 
and Advanced reading achievement levels 
for grades 4, 8, and 12 are presented in 
figures 1.4 through 1.6. The achievement 
levels are cumulative. Therefore, students 
performing at the Proficient level also 
display die competencies associated with 
the Basic level, and students at the Advanced 
level also demonstrate the competencies 
associated with both the Basic and the 
Proficient levels. For each achievement level 
listed in figures 1.4 through 1.6, the scale 
score that corresponds to the lowest score 
within that level on the NAEP reading 
scale is shown in parentheses. For example, 
in figure 1.4 the scale score of 238 corre- 
sponds to the lowest score in the range 
defining the grade 4 Proficient level of 
achievement in reading. 
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Figure 1 .4 Descriptions of NAEP reading achievement levels, grade 4 



Grade 4 

Achievement levels 



Basic 

(208) 


Fourth-grade students performing at the Basic level should demonstrate an understanding of the 
overall meaning of what they read. When reading text appropriate for fourth graders, they should be 
able to make relatively obvious connections between the text and their own experiences, and extend 
the ideas in the text by making simple inferences. 

For example, when reading literary text, they should be able to tell what the story is generally 
about — providing details to support their understanding — and be able to connect aspects of the 
stories to their own experiences. 

When reading informational text, Basic- level fourth graders should be able to tell what the 
selection is generally about or identify the purpose for reading it, provide details to support their 
understanding, and connect ideas from the text to their background knowledge and experiences. 


Proficient 

(238) 


Fourth-grade students performing at the Proficient level should be able to demonstrate an overall 
understanding of the text, providing inferential as well as literal information. When reading text 
appropriate to fourth grade, they should be able to extend the ideas in the text by making 
inferences, drawing conclusions, and making connections to their own experiences. The connections 
between the text and what the student infers should be clear. 

For example, when reading literary text, Proficient-\m\ fourth graders should be able to summarize 
the story, draw conclusions about the characters or plot, and recognize relationships such as cause 
and effect. 

When reading informational text, Proficient level students should be able to summarize the 
information and identify the author's intent or purpose. They should be able to draw reasonable 
conclusions from the text, recognize relationships such as cause and effect or similarities and 
differences, and identify the meaning of the selection's key concepts. 


Advanced 

(268) 


Fourth-grade students performing at the Advanced level should be able to generalize about topics in 
the reading selection and demonstrate an awareness of how authors compose and use literary 
devices. When reading text appropriate to fourth grade, they should be able to judge texts critically 
and, in general, give thorough answers that indicate careful thought. 

For example, when reading literary text, Advanced- level students should be able to make 
generalizations about the point of the story and extend its meaning by integrating personal 
experiences and other readings with ideas suggested by the text. They should be able to identify 
literary devices such as figurative language. 

When reading informational text, Advanced- level fourth graders should be able to explain the 
author's intent by using supporting material from the text. They should be able to make critical 
judgments of the form and content of the text and explain their judgments clearly. 



SOURCE; National Assessment Governing Board. (2002). fading Framework for the 2003 Hatioool Assessment ofFducalional Progress. Washington, DC ; Author. 
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Figure 1 .5 Descriptions of NAEP reading achievement levels, grade 8 



Grade 8 

Achievement levels 



Bask Eighth-grade students performing at the Basic level should demonstrate a literal understanding of what 

(243) they read and be able to make some interpretations. When reading text appropriate to eighth grade, 

they should be able to identify specific aspects of the text that reflect the overall meaning, extend the 
ideas in the text by making simple inferences, recognize and relate interpretations and connections 
among ideas in the text to personal experience, and draw conclusions based on the text. 

For example, when reading literary text, Basic- level eighth graders should be able to identify themes 
and make inferences and logical predictions about aspects such as plot and characters. 

When reading informational text, they should be able to identify the moin idea and the author's 
purpose. They should make inferences and draw conclusions supported by information in the text. They 
should recognize the relationships among the facts, ideas, events, and concepts of the text (e.g. ; 
cause and effect, order). 

When reading practical text, they should be able to identify the main purpose and make predictions 
about the relatively obvious outcomes of procedures in the text. 



Proficient Eighth-grade students performing at the Proficient level should be able to show an overall understand- 

(281 ) ing of the text, including inferential as well as literal information. When reading text appropriate to 

eighth grade, they should be able to extend the ideas in the text by making clear inferences from it, 
by drawing conclusions, and by making connections to their own experiences— including other reading 
experiences. Proficient eighth graders should be able to identify some of the devices authors use in 
composing text. 

For example, when reading literary text, students at the Proficient level should be able to give details 
and examples to support themes that they identify. They should be able to use implied os well as 
explicit information in articulating themes; to interpret the actions, behaviors, and motives of 
characters; and to identify the use of literary devices such as personification and foreshadowing. 

When reading informational text, they should be able to summarize the text using explicit and 
implied information and support conclusions with inferences based on the text. 

When reading practical text, Proficient-\e\e I students should be able to describe its purpose and 
support their views with examples and details. They should be able to judge the importance of certain 
steps and procedures. 




Advanced Eighth-grade students performing at the Advanced level should be able to describe the more abstract 

(323) themes and ideas of the overall text. When reading text appropriate to eighth grade, they should be 

able to analyze both meaning and form and support their analyses explicitly with examples from the 
text, and they should be able to extend text information by relating it to their experiences and to world 
events. At this level, student responses should be thorough, thoughtful, and extensive. 

For example, when reading literary text, Advanced- level eighth graders should be able to make 
complex, abstract summaries and theme statements. They should be able to describe the interactions 
of various literary elements (i.e., setting, plot, characters, and theme) and explain how the use of 
literary devices affects both the meaning of the text and their response to the author's style. They 
should be oble to critically analyze and evaluate the composition of the text. 

When reading informational text, they should be able to analyze the author's purpose and point of 
view. They should be able to use cultural and historical background information to develop perspectives 
on the text and be able to apply text information to broad issues and world situations. 

When reading practical text, Advanced-M students should be able to synthesize information that 
will guide their performance, apply text information to new situations, and critique the usefulness of the 
form and content. 

SOURCE: Nationd Assessmenl Governing Board. (2002). Roodng Framework for the 2003 Hafooal Assessnmtofbkxjfionai Progress. Washington, DC : Airihcf. 
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Figure 1.6 Descriptions of NAEP reading achievement levels, grade 12 



Grade 12 

Achievement levels 



Basic 

(265) 


Twelfth-grade students performing at the Basic level should be able to demonstrate an overall under- 
standing and make some interpretations of the text. When reading text appropriate to twelfth grade, 
they should be able to identify and relate aspects of the text to its overall meaning, extend the ideas 
in the text by making simple inferences, recognize interpretations, make connections among and relate 
ideas in the text to their personal experiences, and draw conclusions. They should be able to identify 
elements of an author's style. 

For example, when reading literary text, flas/r-level twelfth graders should be able to explain the 
theme, support their conclusions with information from the text, and make connections between 
aspects of the text and their own experiences. 

When reading informatianal text, flas/c-level twelfth graders should be able to explain the main idea 
or purpose of a selection and use text information to support a conclusion or make a point. They 
should be able to make logical connections between the ideas in the text and their own background 
knowledge. 

When reading practical text, they should be able to explain its purpose and the significance of specific 
details or steps. 


Proficient 

(302) 


Twelfth-grade students performing at the Proficient level should be able to show an overall understand- 
ing of the text, which includes inferential as well os literal information. When reading text appropriate 
to twelfth grade, they should be able to extend the ideas of the text by making inferences, drawing 
conclusions, and making connections to their own personal experiences and other readings. Connec- 
tions between inferences and the text should be clear, even when implicit. These students should be 
able to analyze the author's use of literary devices. 

When reading literary text, Proficienthve 1 twelfth graders should be able to integrate their personal 
experiences with ideas in the text to draw and support conclusions. They should be able to explain the 
author's use of literary devices such as irony and symbolism. 

When reading infarmotive text, they should be able to apply text information appropriately to 
specific situations and integrate their background information with ideas in the text to draw and support 
conclusions. 

When reading practical text, they should be able to apply information or directions appropriately. 

They should be able to use personol experiences to evaluate the usefulness of text information. 


Advanced 

(346) 


Twelfth-grade students performing at the Advanced level should be able to describe more abstract 
themes and ideas in the overall text. When reading text appropriate to twelfth grade, they should be 
able to analyze both the meoning and the form of the text and explicitly support their analyses with 
specific examples from the text. They should be able to extend the information from the text by 
relating it to their experiences and to the world. Their responses should be thorough, thoughtful, and 
extensive. 

For example, when reading literary text, Advanced level twelfth graders should be able to produce 
complex, abstract summaries and theme statements. They should be able to use cultural, historical, 
and personal information to develop and explain text perspectives and conclusions. They should be 
able to evaluate the text, applying knowledge gained from other texts. 

When reading informational text, they should be able to analyze, synthesize, and evaluate points af 
view. They should be able to identify the relationship between the author's stance and elements of the 
text. They should be able to apply text information to new situations and to the process of forming new 
responses to problems or issues. 

When reading practical text, Advanced\eve\ twelfth graders should be able to make critical 
evaluations of the usefulness of the text and apply directions from the text to new situations. 



SOU RCt National Assessment Governing Board. (2002). boding Framework for the 2003 Nationol Assessment of Educational Progress. Washinglon, DC : Author. 
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Trial Status of 
Achievement Levels 

The 2001 NAEP reauthorization law 
requires that the achievement levels be 
used on a trial basis until the Commissioner 
of Education Statistics determines that the 
achievement levels are “reasonable, valid, 
and informative to the public” 4 Until that 
determination is made, the law requires the 
Commissioner and die Board to state 
clearly the trial status of the achievement 
levels in all NAEP reports. 

In 1993, the first of several concession- 
ally mandated evaluations of the achieve- 
ment level setting process concluded that 
die procedures used to set the achievement 
levels were flawed and that the percentage 
of students at or above any particular 
achievement level cut point may be under- 
estimated. 5 Others have critiqued these 
evaluations, asserting that the weight of 
die empirical evidence does not support 
such conclusions. 6 

In response to the evaluations and 
critiques, NAGB conducted an additional 
study of the 1992 reading achievement 
levels before deciding to use them for 
reporting 1994 NAEP results. 7 When 



reviewing the findings of this study, the 
National Academy of Education (NAE) 
panel expressed concern about what it saw 
as a “confirmatory bias” in the study and 
about the inability of this study to “address 
the panel’s perception that the levels had 
been set too high.” 8 In 1997, the NAE 
panel summarized its concerns with inter- 
preting NAEP results based on the 
achievement levels as follows: 

First, the potential instability of 
the levels may interfere with the 
accurate portrayal of trends. Second, 
the perception that few American 
students are attaining the higher 
standards we have set for them may 
deflect attention to the wrong aspects 
of education reform. The public has 
indicated its interest in benchmarking 
against international standards, yet it 
is noteworthy that when American 
students performed very well on a 
1991 international reading assessment, 
these results were discounted because 
they were contradicted by poor 
performance against the possibly 
flawed NAEP reading achievement 
levels in the following year. 9 



4 No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002). 

5 United States General Accounting Office. (1993). Education Achievement Standards: NAGB’s Approach Yields 
Misleading Interpretations . U.S. General xVccounting Office Report to Congressional Requestors. Washington, DC: 
Author. 

National Academy of Education. (1993). Setting Performance Standards for Achievement: A Report of the National 
Academy of Education Panel on the Evaluations of the NAEP Trial State Assessment: An Eialuation of the 1992 Achieve- 
ment Levels . Stanford, CA: Author. 

6 Cizek, G. (1993). Reactions to National Academy of Education Report. Washington, DC: National Assessment Govern- 
ing Board. 

Kane, M. (1993). Comments on the NAE Evaluation of the NAGB Achievement Levels. Washington, DC: National 
Assessment Governing Board. 

7 American College Testing. (1995). NAEP Reading Revisited: An Evaluation of the 1 992 Achievement Level Descriptions . 
Washington, DC: National Assessment Governing Board. 

8 National Academy of Education. (1996). Reading achievement levels. In Quality and Utility: The 1994 Trial State 
Assessment in Reading. The Fourth Report of the National Academy of Education Panel on the Evaluation of the NAEP Trial 
State Assessment. Stanford, CA: Author. 

9 National Academy of Education. (1997). Assessment in Transition: Monitoring the Nation’s Educational Progress (p. 99). 
Mountain View, CA: Author. 

M CHAPTER 1 • NAEP 2002 READING REPORT CARD 




32 



